Web Survey Bibliography
In interpreting questions, respondents extract meaning from how the information in a questionnaire is shaped, spaced, and shaded. We carried out five experiments using five questions measured on a five point scale to investigate how the effects of visual heuristics affect the answers to survey questions and test whether effective question writing (using numerical and verbal labels) may overrule visual effects. The experiments were fielded in the CentERpanel. This is an Internet-based panel, although there is no need to have a personal computer with an Internet connection. If necessary, equipment is provided by CentERdata (the agency). The recruitment of new panel members is done through a random sample of landline numbers of candidates. The panel is designed to be representative of the Dutch population. We demonstrate that respondents use the “middle means typical” heuristic only in a polar point scale. When numbers 1 to 5 are added to the polar point format and with fully labeled scales, the effect of spacing between response options disappears. In a second experiment we show that respondents are confused when options are presented in an inconsistent order and do not follow the “left and top means first” heuristic. In a third experiment we find that inter-item correlations are higher in a polar point scale when items are presented on a single screen instead of separate screens. The effect gets smaller when numbers 1 to 5 are added to the polar point scale. In a fully labeled scale no effect is found. In a fourth experiment, we found no evidence that respondents use the “up means good” heuristic: the use of a decremental or incremental scale did not result in different answer distributions. In a fifth experiment, we show that respondents use the “like means close” heuristic: the adding of numbers that differ both in sign and value (-2 to 2) and the adding of different shadings of red and green color affects respondents’ answers. The effect of different numbers was apparent in both a polar point and fully labeled format, while the effect of color was only apparent in the polar point format.
Conference Homepage (abstract)
Web survey bibliography (129)
- Social desirability bias in self-reported well-being measures: evidence from an online survey; 2017; Caputo, A.
- Comparability of web and telephone surveys for the measurement of subjective well-being; 2017; Sarracino, F.; Riillo, C. F. A.; Mikucka, M.
- Privacy Concerns in Responses to Sensitive Questions. A Survey Experiment on the Influence of Numeric...; 2016; Bader, F., Bauer, J., Kroher, M., Riordan, P.
- Thinking Inside the Box Visual Design of the Response Box Affects Creative Divergent Thinking in an...; 2016; Mohr, A. H.; Sell, A.; Lindsay, T.
- Detecting Insufficient Effort Responding with an Infrequency Scale: Evaluating Validity and Participant...; 2016; Huang, J. L.; Bowling, N. A.; Liu, Me.; Li, Yu.
- Detecting careless respondents in web-based questionnaires: Which method to use?; 2016; Niesen, A. S. M.; Meijer, R. R.; Tendeiro, J. N.
- Eye-tracking Social Desirability Bias; 2016; Kaminska, O.; Foulsham, T.
- Short and Sweet? Length and Informative Content of Open-Ended Responses Using SMS as a Research Mode; 2016; Walsh, E.; Brinker, J. K.
- Participant recruitment and data collection through Facebook: the role of personality factors; 2016; Rife, S. C.; Cate, K. L.; Kosinski, M.; Stillwell, D.
- Quantifying Under- and Overreporting in Surveys Through a Dual-Questioning-Technique Design. ; 2016; de Jong , M.; Fox, J.-P.; Steenkamp, J. - B. E. M.
- Will They Stay or Will They Go? Personality Predictors of Dropout in Online Study; 2016; Nestler, S.; Thielsch, M.; Vasilev, E.; Back, M.
- Development of a scale to measure skepticism toward electronic word-of-mouth; 2016; Zhang, Xia.; Ko, M.; Carpenter, D.
- Psychological research in the internet age: The quality of web-based data; 2016; Ramsey, S. R.; Thompson, K. L.; McKenzie, M.; Rosenbaum, A.
- Internet Abusive Use Questionnaire: Psychometric properties; 2016; Calvo-Frances, F.
- Equivalence of paper-and-pencil and computerized self-report surveys in older adults; 2016; Weigold, A.; Weigold, I. K.; Drakeford, M. K.; Dykema, S. A.; Smith, C. A.
- A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel...; 2016; Golden, L.; Albaum, G.; Roster, C. A.; Smith, S. M.
- Swapping bricks for clicks: Crowdsourcing longitudinal data on Amazon Turk; 2016; Daly, T. M.; Nataraajan, R.
- A reliability analysis of Mechanical Turk data; 2016; Rouse, S. V.
- Quota Controls in Survey Research.; 2016; Gittelman, S. H.; Thomas, R. K.; Lavrakas, P. J.; Lange, V.
- Exploring Factors in Contributing Student Progress in the Open University; 2016; Arifin, M. H.
- Incentive Types and Amounts in a Web-based Survey of College Students; 2015; Krebs, C.; Planty, M.; Stroop, J.; Berzofsky, M.; Lindquist, C.
- Comparison of telephone RDD and online panel survey modes on CPGI scores and co-morbidities; 2015; Lee, C.-K.; Back, K.-J.; Williams, Ro. J.; Ahn, S.-S.
- Enhancing Response Usability in a Web-based Survey, But Did Anyone Use It?; 2015; Yoder, R.
- Equivalency of Paper Versus Tablet Computer Survey Data; 2015; Ravert, R. D.; Gomez-Scott, J.; Donnellan, M. B.
- Development and Validation of a Scale for Social Exhibitionism on the Internet (SEXI); 2015; Vetter, M.; Eib, C.; Hill-Kloss, S.; Wollscheid, P.; Hagemann, D.
- Recruiting for addiction research via Facebook; 2015; Thornton, L. K.; Harris, K.; Baker, A.; Johnson, M.; Kay-Lambkin, F. J.
- Internet Research in Psychology; 2015; Gosling, S. D., Mason, W.
- The Prostate Cancer Journey Results of an Online Survey of Men and Their Partners; 2015; O'Shaughnessy, P. K., Laws, T. A., Esterman, A.
- Twelve-month prevalence and predictors of self-reported suicidal ideation and suicide attempt among...; 2015; Kang, E. H., Kim, G. M., Hyun, M. K., Choi, S. M., Kim, J. M., Woo, J. M.
- Emotion management in online groupwork reported by Chinese students; 2014; Xu, J., Du, J., Fan, X.
- Quality of physical therapy from a patient's perspective; factor analysis on web-based survey data...; 2014; Scholte, M., Calsbeek, H., Nijhuis-van der Sanden, M. W. G., Braspenning, J.
- Discriminating the Effects of Cannabis sativa and Cannabis indica: A Web Survey of Medical Cannabis...; 2014; Pearce, D. D., Mitsouras, K., Irizarry, K. J.
- Awareness and Treatment of Alcohol Dependence in Japan: Results from Internet-Based Surveys in Persons...; 2014; Taguchi, Y., Takei, Y., Sasai, R., Murteira, S.
- Awareness and correlates of the role of physical activity in breast cancer prevention among Japanese...; 2014; Miyawaki, R., Shibata, A., Ishii, K., Oka, K.
- Differences in intrapersonal and interactional empowerment between lurkers and posters in health-related...; 2014; Petrovcic, A., Petric, G.
- Evaluating mixed-mode redesign strategies against benchmark surveys: the case of the Crime Victimization...; 2014; Klausch, L. T., Hox, J., Schouten, B.
- Confirmation Bias in Web-Based Search: A Randomized Online Study on the Effects of Expert Information...; 2014; Schweiger, S., Oeberst, A., Cress, U.
- The Use of Paradata to Predict Future Cooperation in a Panel Study; 2014; Funke, F., Goeritz, A.
- The impact of New Zealand's 2008 prohibition of piperazine-based party pills on young people'...; 2013; Sheridan, J., Dong, C. Y., Butler, R., Barnes, J.
- PRM144 – An adaptable methodology for the design, implementation and conduct of a web-based survey...; 2013; Yeomans, K., Kawata, A. K., Bassel, M., Burk, C. T., Daniels, S. R., Wilcox, T. K.
- The internet user profile of Italian families of patients with rare diseases: a web survey; 2013; Tozzi, A. E., Mingarelli, R., Agricola, E., Gonfiantini, M., Pandolfi, E., Carloni, E., Gesualdo, F.,...
- Education in the Responsible Conduct of Research in Psychology: Methods and Scope; 2013; DiLorenzo, T. A., Becker-Fiegeles, J., Gibelman, M.
- Relative Mode Effects on Data Quality in Mixed-Mode Surveys by an Instrumental Variable; 2013; Vannieuwenhuyze, J. T. A., Revilla, M.
- Reaching and Hearing the Invisible: Organizational Research on Invisible Stigmatized Groups via Web...; 2013; Trau, R. N. C., Haertel, C. E. J., Haertel, G. F.
- On-line questionnaire completion time and personality test scores; 2013; Furnham, A., Hyde, G., Trickey, G.
- Why are you leaving me?? - Personality predictors of answering drop out in an online-study; 2013; Thielsch, M., Nestler, S., Back, M.
- The association between online gaming, social phobia, and depression: an internet survey; 2012; Chen, M.-H., Huang, P.-C., Bai, Y.-M., Wei, H.-T.
- P02.04. Internet survey confirms strong interest in Yoga among fibromyalgia patients; 2012; Carson, J., Bennett, R., Jones, K., Mist, S.
- GRE® program announces big benefits and big savings for GRE® test takers worldwide; 2011
- Going online with assessment; 2011; Burke, E., Mahoney-Phillips, J., Bowler, W., Downey, K.